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ABSTRACT 

Under the sponsorship of the U. S. Office of 
Education, Sand is conducting a several-year study of federally 
funded programs designed to introduce and .spread innovative practices 
in public schools. These change agent programs normally offer 
temporary federal funding to school districts as "seed money" or 
"risk capital." If an innovation is successful, it is assumed that 
the district will continue part or all of the project using other 
sources of funds, and also disseminate it to other schools in the 
district, as well as to other districts that may be seeking change. 
The Rand study examines four such federal programs— Elementary and 
Secondary Education Act Title III, Innovative Projects; Elementary 
and Secondary Education Act Title Vil, Bilingual Projects; Vocational 
Education Act, 1968 Amendments, Part D, Exemplary Programs; and the 
Right-To-Read Program. The study identifies what tends to promote 
various kinds of changes in the schools and what doesn f t. In 
particular, the Rand study will identify for federal, state, and 
local policy makers the nature, permanence, and extent of 
dissemination of innovations that are associated with the various 
federal programs and with various federal, state, and local 
practices. This part tries to distill the study^ methods and results 
for a general audience. (Author) 
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PREFACE 



" Under the sponsorship of the U.S. Office of Education, Rand is conducting a 
several-year study of federally funded programs designed to introduce and spread 
innovative practices in public schools. These change agent programs normally offer 
temporary federal funding to school districts as "seed money" or "risk capital." If 
an innovation is successful, it is assumed that the district will continue part or all 
of the project using'other sources of funds, and also disseminate it to other schools 
in the district, as well as to other districts that may be seeking change. The Rand 
study examines four such federal programs— Elementary and Secondary Education 
Act Title III, Innovative Projects; Elementary and Secondary Education Act Title 
VII, Bilingual Projects; Vocational Education Act, 1968 Amendments, Part D, Ex- 
emplary Programs; and the Right-To-Read Program. The study identifies what tends 
to promote various kinds of changes in the schools and what doesn't; in particular, 
the Rand study will identify for federal, state, and local policymakers the nature, 
permanence, and extent of dissemination of innovations that are associated with the 
various federal programs and with various federal, state, and local practices. 

A series of five reports describes the results of the first year of the Rand study 
(July 1973 to July 1974). Volume I (R-1589/1-HEW, A Model of Educational Change) 
provides a theoretical perspective for the Rand study by analyzing the current state 
of knowledge of planned change in education and by proposing a conceptual model 
of factors affecting change processes within school districts. 1 

Volume II of the series (R-1589/2-HEW, Factors Affecting Change Agent Pro- 
jects) contains the analysis of survey data collected from a national sample of 293 
projects in 18 states during November and December 1973. 

Volume III (R-1589/3-HEW, The Process of Change) summarizes the results of 
29 case studies of change agent projects conducted by Rand staff members and 
consultants in 25 school districts during April and May 1974. These case studies 
were chosen from the original sample of 293 projects initially surveyed. Volume III 
also describes the role of state education agencies in selecting, monitoring, and 
disseminating the change agent projects. 

Four technical appendixes to Vol. Ill describe in detail the federal program 
management approach, state education agency participation, and case studies for 
each of the programs in the study: Title III, App. A; Reading, App. B; Bilingual 
Education, App. C; and Career Education, App. D. Appendix A should be of particu- 
lar interest to researchers or practitioners concerned with the introduction of new 
approaches to classroom instruction. 

Volume IV (R-1589/4-HEW, The Findings in Review) summarizes the findings 
of Vols. I, II, and III, and also synthesizes extensive data collected by Rand on 
federal-level program strategy and management for each of the change agent pro- 
grams. Volume IV includes a discussion of alternative federal strategies for promot- 
ing innovation. 

' Because of Rand's interest in advancing knowledge of organizational behavior in educational insti- 
tutionMhe r^ch underlying this report was supported in part by an allocat.on of Rand corporate 
research flunds. ^ 
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This report, Vol. V, tries to distill the study's methods and results for a general 
audience. — — — 

Subsequent research will collect additional data on Titles III and VII of ESEA, 
particularly those projects for which federal funding has expired. 
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L INTRODUCTION 



The Rand change agent report, commissioned by the U.S. Office of Education 
(USOE), studies four federally sponsored programs that are trying to promote educa- 
tional change in the public schools by paying for the costs of innovative projects for 
a trial period. The study aims to help improve the ways that policies are made and 
carried out by describing how the process of innovation operates for these projects 
and by trying to discern what factors affect their outcomes. 

This may sound simple enough, but it is not. American public education works 
through a complex system of influences. First, there is the formal system that 
finances the schools, each part of it with a certain influence: school district staff and 
school boards propose local contributions, which local voters pass upon; educational 
interest groups and state education agencies propose state contributions, which 
state legislatures and governors pass upon; national educational interest groups and 
federal education agencies propose federal contributions, which Congress and the 
President must approve. Second, there are formal structures of administration: 
teachers, principals, district staff, school boards, county and state educational agen- 
cies, federal education agencies, each with varying degrees of authority over differ- 
ent elements, of the system. Third, there is a complex structure of informal influence 
and authority: in the community, there are parents, community groups, and com- 
munity power structures; in the school, there are varying relations among students, 
teachers, parents, the principal, and district administrators; in the school district, 
there are relations among schools, central administration, school boards, and the 
social and political influences of the community. Beyond the school district, there 
are informal influences at state and national levels— for example, the influence of 
state and national organizations of teachers and administrators, policies of teacher 
training institutions, state and national standards for college admission, pressures 
from textbook publishers. Finally, at all levels, there are constantly shifting pres- 
sures about values, leading to consensus and conflict over educational aims and the 
distribution and focus of educational resources. 

Within this complex system, it is no mean task to .find out how change processes 
work for federally funded programs, or to isolate the different factors that affect 
project outcomes. We regard this series of reports as a first step toward a goal that 
is remote and difficult to reach, but nonetheless essential as one basis Tor effective 
educational reforms. 

This volume is a summary of the first year of Rand's work (July 1973 to July 
1974). It also presents in Sec. V some preliminary ideas about what the first year's 
findings may mean for the future of educational reform. The second phase of the 
work, covering the 1974-75 and 1975-76 school years, will speak to the policy ques- 
tions in more detail. By then we will have the benefit of more experience in finding 
out how innovative projects are continued and spread after federal support has 
ended. 

Section II of this volume describes the background of the study and sets forth 
the study design. Section III discusses the theoretical basis for the approach taken 
in the Rand study. Section IV reviews the findings of the first year, and Sec. V tells 
what we think our work may mean for policy. The Appendix gives the contents of 
the other volumes in this series. § 
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IL BACKGROUND 



Before the 1950s, the government of the United States almost never interfered 
directly in the curriculum of the public schools. 1 In this century, the biggest federal 
influence was, and still remains, the action of the Supreme Court in Brown v. Topeka 
Board of Education (1954) and later cases, which in effect said that separating / 
students by race in public schools is unconstitutional These decisions were moment 
tous, but they remained in the federal tradition and were not aimed directly ,at 
either the curriculum or the educational priorities of the schools. 

But by the late 1950s, the White House and Congress began to enter the^rena 
of public schools on a large scale. The National Defense Education Act df 1958 
(NDEA) and the Elementary and Secondary Education Act of 1965 (ESEA), stimulat- 
ed respectively by Soviet advances in space technology and the civil Rights and 
anti-poverty movements of the 1960s, defined a new federal role in schooling. It was 
based on large federal support payments programs that were intended to meet the 
goals of these new laws: subsidizing special new curricula and materials, providing 
better education for the children of the poor, training young people for careers, 
encouraging higher education for minority groups and poor people, and encouraging 
innovation in the public schools, which is the subject of this study. About one-tenth 
of federal aid to public schools, which now adds up to more th^n $3.5 billion a year, 
is spent to promote innovations. The USOE pays out most of these funds for a 
number of categorical programs, including those covered ih this study. 2 

t> 

• Elementary and Secondary Ediffiation Act Title III, Innovative Projects 
($120-^190 million annually in recent years)/ 

• Elementary and Secondary Education Act, Title VII, Bilingual Projects 
($45-$85 million) 

• Vocational Education Act, 1968 Amendments, Part D, Exemplary Pro- 
grams ($16 million) 

• Right-To-Read ($12 million) 



THE PROGRAMS 

The four programs have much in common, even though each has different aims 
and methods. Each provides funds, intended to be "seed money," to many school 
districts for support of innovative projects for a few years in each district. Not only 
are the change agent program methods alike, but so are the underlying assumptions: 

• American schools will benefit from new methods, because the schools are 
not serving everyone's educational needs well enough now. 

1 Except; starting in 1921, to fund vocational education in the schools. 

* Other federal programs not included in this study also aim at encouraging educational innovations 
(e g , certain programs for handicapped students, experimental schools, educational voucher demonstra- 
tions, Follow Through, elements of the Emergency School Assistance Act and of the Head Start legisla- 
tion). Ck -» 
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• These reforms can be brought about by temporary federal support of pro- 
jects at the school level. No major prior changes in district bureaucracy or 
in the broader social system are necessary in order forprojects to be suc- 
cessful. 

• The best (or most feasible) way to bring about change in education is by 
putting money into the public schools, instead of some other way (such as 
television, subsidizing co \peting private schools, reducing public support 

' for public schools, or training parents to be educators). 

• Changes sought by these programs would not come about fast enough, or 
perhaps not at all, without federal seed money. 

In other words, these programs all take the existing framework of society and 
schooling as a fact of life; they choose to use school-based projects as the vehicles for 
broader educational and social change. 

Despite the similarities, the four programs are also very different— each focuses 
on a different aspect of change and uses different management strategies toward its- 
goals. 

Title III of ESEA gives three-year grants (sometimes extended to a fourth year 
to encourage the spread of successful projects) to projects that introduce new meth- 
ods or spread proven methods to new sites. The grants are awarded to local education 
agencies (LEAs) through competitions, supervised in each state by the state educa- 
tion agency (SEA). Du^ig the period 1971-1975, the federal government reserved 
15 percent of Title frl %nds for competitive awards to be made directly to school 
districts, which lately are often used for disseminating successful Title III projects 
to new sites. 

Right-To-Read, which aims at the eventual elimination of illiteracy, encourages 
SEAs and school districts to pay more attention to reading, particularly for poor 
children. The Right-To-Read demonstration projects, the program component ad- 
dressed in this study, fund LEAs to use the Right-To-Read planning and manage- 
ment approach to carrying out reforms in reading instruction. 

Vocational Education, Part D, 3 offers three-year grants for exemplary projects 
that are intended to increase students' awareness of and readiness for post-school 
careers. Half the money is spent directly by USOE, usually on one project in each 
state; the other half goes to the SEAs, which choose other exemplary projects. The 
aim of the program is to find new ways of bridging the gap between school and 
earning a living. 

Title VII of ESEA (bilingual education) offers five-year grants, administered by 
USOE, to help educate children from foreign-language backgrounds whose English 
is weak, or whose cultural values seem to be threatened by the dominant standards 
of American life. The program has strong political support from people of Spanish- 
language origin. 

These four programs are just a sample of many that have received federal 
support over the past decade; in addition, most LEAs and SEAs have used their own 
funds to try out new methods in education. A good deal of time and money has been 
devoted to evaluating these efforts, to find out what they accomplished and how 
useful their results might be for other schools and districts. The evaluation research, 

» The Vocational Education Act, 1968 Amen jjr^jts, Part D, will be referred to hereafler as VEA Part 



whose claims to validity are plagued by profound weaknesses in measurement meth- 
ods, points to rather discouraging general findings: (1) some projects have helped 
improve markedly some students' skills, behavior, or attitudes, but successful pro- 
jects are hard to export; (2) few if any projects are consistent — even the most success- 
ful ones work well only at a particular time or place, or for some students and not 
for others: 

These results have raised serious questions about the effectiveness of new meth- 
ods, and, in particular, about the usefulness of federal efforts to promote innovation 
in the schools. 



STUDY DESIGN 

In light of these findings and of the federal government's own questions about 
the assumptions underlying change agent programs, USOE asked Rand in June 
1973 to study these four programs, focusing on the following questions: 

• How should public school innovations and their dissemination be as- 
sessed? A 

• How do school districts choose, carry out, and pass on different kinds of 
innovations? 

• How do local differences, project differences, or differences in federal pro- 
grams (e.g., differences in target groups, resource use, project strategies, 
bureaucracy, politics, federal program strategies, or the interaction of all 
of these factors) affect how projects are chosen and carried out? 

• What should the federal government do about educational innovation in 
a situation where federal authorities have, by law, custom, and political 
tradition, little direct control over public schools, and are not prepared to 
spend large sums of money to influence their behavior? 

To examine these questions — in other words, to find out more about the process 
^innovation and about what factors affect the outcomes of innovative projects — we 
dev6ho]5e3 a research design for the first year of the study, which included literature 
review, development of theory, data collection, analysis of data, and preparation of 
the first-year reports. The tasks, described fully in App. A of Vol. IV, were as follows: 

1. An extensive review of the literature on educational innovations, leading 
to development of a theoretical approach toward the subject, described in 
Vol. I (A Model of Educational Change by Paul Berman and Milbrey Wal- 
lin McLaughlin), which serves as part of the basis for the data collection 
and analysis. 

2. A nationwide survey in 18 states of 293 change agent projects, each hi its 
last or next to last year of federal funding. The survey, conducted for Rand 
by the National Opinion Research Center in December 1973 and January 
1974, included personal interviews with some 1735 people at all levels in 
the school district, from superintendent to classroom teacher. It sought to 
find out what factors influence the outcomes of change agent projects. The 
findings of that survey are described and analyzed in Vol. II, Factors Affect- 
ing Change Agent Projects by Paul Berman and Edward W. Pauly. 

3. Field studies conducted in April and May 1973 by Rand staff at 29 projects, 
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drawn from the survey sample. The staff observed what was going on in the 
projects and interviewed project staff in order to understand how the 
change process worked in different places, for different projects, supported 
by different federal programs. Volume III, The Process of Change by Peter 
W, Greenwood, Dale Mann, and Milbrey Wallin McLaughlin, discusses the 
results of that work. Four separately bound appendixes to Vol. Ill present 
the field study details for the 29 projects (App. A, classroom organization 
and staff development; App. B, reading; App. C, bilingual education; App. 
D, career education). 

4. Rand staff interviews with federal and SEA officials who work on the four 
change agent programs. These included telephone interviews with 54 SEA 
officials in 18 states, visits to 9 SEAs for more detailed personal interviews, 
and a series of personal interviews with officials at USOE and the Depart- 
ment of Health, Education, and Welfare (DHEW). The appendixes to Vol. 
HI contain brief summaries of this work. 

5. Writing a first-year final report in fi/e volumes. Volumes I, II, III, and V 
(this volume) are described above. Volume IV, The Findings in Review by 
Paul Berman and Milbrey Wallin McLaughlin, draws all this work to- 
gether; it relates the findings of the survey and field studies to the theory 
and suggests some conclusions about federal programs and about policies 
supporting educational change. 4 

The study design for the second phase, summarized in App. A of Vol. IV, will 
look at what happens to projects in the two largest change agent programs when 
federal funding stops. The work will again be based on surveys and field studies and 
will include Title III and Title VII projects that were visited during 1973-74. the 
second phase of the work will test the first-year findings and will also allow us to 
test hypotheses that were not explored during the first year. 

* The Appendix to this volume gives the tables of contents of Vols. MV. 
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III. THE THEORETICAL MODEL 



Before going on to describe the findings of the first-year data analysis and 
fieldwork, we set forth here the hypotheses about educational ghange that are the 
% intellectual foundation of our work. 1 In this study we had to develop some theory 
as part of the stu^y because existing theories ofsocial change did not 5 f 41 *" ~nse 
of the American educational system to change agent programs. Likr » . . i his 
first-year report, the development and testing of the theoretical mo.*., v still far 
from definitive. 

If there is no applicable theory, why should Rand try to develop one as part of 
its study? Why not simply collect data and analyze the results to see which projects 
were successful or unsuccessful and which ones were or weren't continued or spread 
after federal funding stopped? The answer is simple — this is a policy study, and if 
USOb, S^As, and LEAs seek innovation, they will look (if possible) for policies'that 
promoteitheir goals. But it is much easier to find such policies if we can provide an 
orderly and logical description of How change occurs. This description, if validated, 
. can serve as a preliminary theory of change and can thereby offer some guidance 
to policymakers in making choices that affect the complex structure of American 
education. This process of developing and testing theory is particularly important 
when there is no clear connection between policies and results, when goals them- 
selves are unclear, when the means or technologies used to promote change are hard 
to describe and vary over time and place, and when the policies being studied are 
often only a small factor in the many forces that affect outcomes. All of these 
conditions apply to efforts at public school reform, thereby leaving federal policy- 
v makers in the difficult position of following policies for change without ever being 
sure what connection there is between the policies and the goal£ of policy. 

The Rand study began with a review of the literature on educational innova- 
tions and on theories of educational change. Most literature on educational effective- 
ness agrees that few innovative projects, when compared with traditional practices, 
consistently improve student outcomes. Furthermore, successful projects usually 
don't transfer well. What worked well at one time and place may not work at all 
at another time or in a different setting. 

But [laymen and experts alike disagree about why this is so. Some say that since 
schools are already doing as much as can be expected, new practices are unlikely 
to improve outcomes. Others claim that the new ideas that have been tried out so 
far don't amount to much because there has been too little research and develop- 
ment. A third view is that real changes are taking place but are impossible to detect 
through evaluation, either because customary measures of outcome (such as stand- 
ardized achievement tests or tests that try to measure changes in children's atti- 
tudes toward themselves and others) are deficient or because the government and 
the public are in too much of a hurry — educational change takes place slowly 
throughout the system. Finally, some people believe that the educational system 
itself resists real innovation, adopting many new practices but carrying them out 
largely as new ways of doing the same thing. These people claim that successful 

13 
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innovation requires more understanding of how school systems behave and also 
requires changes in school systems that will encourage genuine and lasting innova- 
tion. 

Faced "'<th these competing explanations, we turned to the existing body of 
theory • nee, particularly to theori ;s of planned change in organizations. 
These t •• . tend to focus on how a rational research and development process 
would interact with a clientele of research "customers" leading to efficient develop- 
ment of innovations and their adoption and dissemination throughout the market. 2 
But these theories do not apply well to public education. The educational system is 
different from the agricultural system or the health care system. There is no market- 
place where customers can choose to buy one firm's products or services instead of 
another's. School districts may adopt innovations for opportunistic reasons (e.g., the 
desire to obtain federal grants) or from problem-solving motives (the desire to 
change an existing or anticipated situation). However, once the innovation is adopt- 
ed, it has to fit into a system where there is nothing in the situation that forces the 
educational "fh;m" either to carry out the innovation or else to lose its customers. 
The public schools are tax-supported and will continue to operate whatever the fate 
of the innovation. Furthermore, each school and school system is somewhat different 
from others and there is no clear and invariate technology. A new curriculum is not 
like a new lathe or a new variety of wheat because the curriculum is adapted by 
teachers, students, and administrators to fit their needs and perceptions. As a result, 
educational innovations are carried out, or not carried out, in different ways at 
different times or places. 

Therefore, without playing down the importance of measurement problems or 
the other explanations for apparent ineffectiveness, we concluded that for a study 
of educational change it was important to focus on two aspects: (1) developing 
hypotheses about the innovative process that would cast light on implementation— 
the way that in novations at ,arried out in the setting of the schools; (2) developing 
an approach that would help to describe which factors affect project outcomes. 

The first aspect assumes that the innovative process passes through distinct 
stages and that the institutional setting (the school, the school district, the commun- 
ity) has a profound influence on Jhe fate of innovations. The second aspect assumes 
that certain identifiable factors may account for a significant amount of the differ- 
ences in project outcomes. 

• See Vol. I. 
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IV. FINDINGS 



We have reviewed the background of this study and the theoretical approach 1 
upon which the research is based. This section begins by describing the stages of the 
change process, identifying the outcomes of interest, and defining the factors that 
affect those outcomes. It then summarizes the findings of research: how the process 
of change operated in the projects we observed, and what factors affected project 
outcomes. 

This is an interim report, and we defer a broader discussion of the implications 
of these findings to the final phase of the study. This summary is intentionally 
compressed. Readers who seek a more complete explanation of methods, evidence, 
and findings should turn to Vols. II and III, which describe the research, and to Vol. 
IV, which offers a synthesis of the findings. 



DEFINING THE STAGES OF INNOVATION AND THE 
FACTORS AFFECTING OUTCOMES 

The Stages of Innovation 

We found it useful to define innovation as a three-stage process of initiation, 
implementation, and incorporation. 

Initiation is the period when the LEA staff conceives and plans possible-innova- 
tions, looks for resources, and decides which projects to select for support. The kind 
of support and commitment that the project receives then casts a long shadow over 
the implementation process that follows. 

Implementation is the stage where the project, as a reality, first confronts anoth- 
er reality — the institutional setting of the school and the district. This stage, where 
plans must be translated into practice, is decisive. Rarely are projects carried out 
exactly to the letter of the original design. Instead they must be adapted into the 
institutional setting, while the people in the schools, and.the organizations those 
people have created, must at the same time adapt to the.demands of the project. In 
other words we hypothesize that innovation in schools is a process of mutual adapta- 
tion. If that adaptation does not take place, then the prospects for effective im- 
plementation are dim. This hypothesis is consistent with our observations about the 
absence of market pressures on schools and the ineffectiveness of technologies that 
v attempt to impose themselves automatically on the adopter, and with the abstract 
character of most projects' goals in combination with uncertain relationships be- 
tween these goals and the nieans proposed to reach them. Given this situation, it is 
very hard to know whether there has been any real innovation. One way to find out 
is to see whether there has been a change in what goes on in the system. Therefore, 
we define implementation as the change process that occurs when the project and 
the system interact. If nothing happens to change the project, then it probably never 

1 See also Vol. L 
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really "met" the system. If nothing happens to change the setting, then there 
probably was no real implementation. 

Incorporation, the final stage in innovation, reflects the decisions that are made 
when the project is no longer an exotic plant, nurtured by special funding, but 
becomes part of the daily environment of the district. This is an ultimate test of the 
district's commitment, for now the initial venture becomes a new orthodoxy, one 
that the district must support with its own funds. In this stage, the district decides 
whether to support all or part of the innovation and whether that support should 
be districtwide or confined to the original site or to volunteer sites or have some 
other selective arrangement. We hypothesize that incorporation is based on a Varie- 
ty of economic, political, and social factors that affect the district, which in the 
decisionmaking process often dwarf the issue of project success or failure. That is, 
many successful projects are likely to be discontinued and unsuccessful ones incorpo- 
rated, for example, because of the presence or absence of effective community or staff 
lobbying, for financial reasons unrelated to project success, or on other grounds. 
Furthermore, incorporation may take place at the classroom level without any 
conscious LEA decision. If teachers find part or all of the innovation to be useful, 
they can often n;ake it part of the classroom routine without any need for district 
support, at least at the change agent project site. 2 

Each of these three stages engages a partially different group of people whose 
influence on the respective decisions waxes and wanes according to the stage. Thus, 
for example, the USOE career education staff is very important in the initiation 
decision for federal VEA Part D grants, but much less important than classroom 
teachers, principals, or project directors in affecting the process of implementation. 
However, continuation is largely a decision for the senior administrators of the 
district, once federal funding stops. 

Factors Affecting Outcomes 

Understanding the process of change is a hard task, but it is only part of the 
story. In order to assess innovations, there must be some, ways to measure project 
outcomes and to find out what affects them. For the change agent study, where 
projects were new and the different programs had very diverse objectives, we could 
not rely on measures of student outcomes, such as changes in behavior or test scores. 
There was no practical way to compare, for example, student outcomes in bilingual 
reading projects with those in remedial mathematics; even within these categories, 
it was often too early in the life of the project to assess differences in student 
outcomes; and well-known measurement problems made such an approach even 
more dubious. 

Therefore, we studied three other types of outcomes, using the information 
gathered from the survey and case studies. These outcomes were measured by five 
indicators from the survey data (which were also observed in the case studies); their 
shorthand names are shown in parentheses. 

Implementation Outcomes 

1. The relative extent to which participants believed that project goals were 
achieved (perceived success). 



a Spreading the innovative methods to new sites in the district may often 'call for administrators' 
approval or at least neutrality. 4 
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2. The type and extent of change in teacher and administrator behavior as 
perceived by participants (change in behavior). 

3. The extent to which implementation followed project design (fidelity of 
implementation). 

Continuation Outcome 

4. The extent of LEA support for the project after federal funding stops, as 
reported by superintendents (expected continuation). 

Dissemination Outcome 

5. The extent of project diffusion to other schools or to other districts (dis- 
semination). This indicator was not used in the first year's work, but will 
be an important outcome for the 1974-75 and 1975-76 research. 

The three implementation outcomes are designed to measure what school and 
district staff thought about the project's success. If the staff perceived, for example, 
that the project brought little change ; .n their own behavior, and if there v/as little 
relationship between original project design and what happened in the schools, then 
we could conclude that there had been little or no mutual adaptation— the schools 
simply conducted business as usual and modified the project accordingly. The con- 
firmation outcome is supposed to measure how much, if any, of the project's methods 
seemed important enough to the district to merit long-run local support. The dis- 
semination outcome measures the extent of this local support (does the district use 
its energy and resources to spread project methods to other schools and classrooms 
in the district?) and also measures how much and under what conditions innovations 
spread, wholly or partly, from one district to others. 

This report deals almost entirely with projects that were still receiving federal 
funds in 1973-74. Therefore, these volumes focus on implementation outcomes and 
expected continuation. The next phase of the work, to be reported in 1976, will 
examine what happens after federal funding stops, and will therefore be more useful 
in assessing actual experience with continuation and dissemination. 

We believe that three kinds of factors are most likely to affect implementation, 
continuation, and dissemination: 

• .Project characteristics (amount of funding, educational methods, im- 
plementation strategy, scope of change). 

• Federal policies (programs' aims and management strategies). 

• Institutional setting (organizational climate and motivation of staff, char- 
acteristics of the school and district, and characteristics of the people most 
closely involved with the project). 



RESEARCH FINDINGS 

Describing the Process of Innovation 

In the three stages of the change process — initiation, implementation, and in- 
corporation—what elements stood out? 

41 
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In the initiation stage, there seemed to be two types of motives at work- 
opportunism (where the district's quest was primarily for federal funds) and prob- 
lem-solving (where the district, sensing real needs for change, tried to deal system- 
atically with its own adaptation to present realities and future prospects). Depend- 
ing on which motive was dominant, the implementation process and therefore 
project outcomes were profoundly affected. In general, whatever the motive, project 
treatments or technologies did not follow the "rational" research and development 
* model. 3 Instead they tended to rely on local knowledge; projects installed in toto 
from outside the district usually failed to gather LEA staff support. Early staff 
involvement in project planning and design was usually a component of effective 
implementation. 

Implementation never consisted of merely applying a fixed technology'in an 
unchanging setting. The interaction between project and setting was neither auto- 
matic nor certain. Three types of results emerged: mutual adaptation, when both 
the project design and the institutional setting changed; nonimplementation, when 

\ neither element changed, usually because of LEA indifference or lack of ability; and 
. cooptation, where the LEA changed the project, usually emasculating it, to meet its 
needs, without any correspondng change in the institution. 

V * * JProjects that showed evidence of mutual adaptation were most likely to be 
^%tively implemented and to persist. Mutual adaptation did not take place in the 
opportunistic projects we observed. A problem-solving approach did seem, to be 
necessary in order for mutual adaptation to occur, but such projects were sometimes 
themselves unsuccessful. For example, in one case a project did not do well when 
district administrators sought to bring about changes in management or staffing 
patterns that the staff either could not handle or resisted for other reasons. In 
practice, mutual adaptation was shown by various adjustments in project and set- 
ting, such as modifications in project goals or unanticipated changes in staff relation- 
ships. Mutual adaptation tended to be more extensive when projects were complex, 
treatments were relatively hard to prescribe in advance, and projects required 
substantial changes in staff behavior— for example, in open-classroom projects, 
where teaching methods, teacher-student relations, and classroom facilities were all 
changed together. In practice, this meant that mutual adaptation was difficult to 
achieve because it imposed simultaneous demands for flexibility and change. There- 
fore, it was often necessary to develop implementation strategies that promoted 
mutual adaptation. 

Incorporation had two aspects. At the classroom level, teachers and principals 
often said that they would plan to assimilate elements of the change agent project 
into the regular routine of project' classroom or school, whether or not the district 
authorized any formal continuation of the project. The assimilation of new practices 
was more likely when the project replaced existing practices, when there was an 
emphasis on training, when teacher training focused on practical classroom issues, 
and when curriculum materials were developed by the project staff. At the district 
level, the superintendent might make a conscious decision to continue the project 
and encourage its incorporation into the practice of some or all schools in the 
district. If the superintendent thought a project was not too expensive, was central 
to the district's educational priorities, had the support of the staff, and was reason- 

* See Vol. I and p. 7 of this volume. 

18 



ERIC 



12 



ably "successful" (if not now at least in the long run), then the project was likely 
to be continued by the district. 

Federal policies had the greatest effect at the initiation stage, mostly by pro- 
viding funds and encouragement for LEAs to undertake projects that reflected 
federal priorities; and to a lesser extent, by basing awards on certain policies and 
guidelines that affected LEA project design choices (see Vol. IV). Federal policies Had 
little direct effect on implementation. There were indirect effects, because each 
federal program was associated with different implementation problems, as dis- 
cussed below. Federal policies also had little direct effect on expected continuation- 
it was primarily the LEA's decision. 4 Such decisions naturally depended on how 
LEA officials saw the project— whether it was (1) "successful," (2) affordable, (3) 
^important to the district's priorities, and (4) politically acceptable. If the answer was 
yes to most of those, points, then it was likely to be continued. In the case of opportu- 
nistic projects, some of the answers were usually negative; in the case of problem- 
solving projects, they were often positive. In other words, the motives that underlay 
continuation decisions tended to be the same as those that led to initiation decisions. 

■> 

Findings about Factors Affecting Project "Outcomes" 

Our study design hypothesized 5 that three kinds of factors— federal policies, 
project characteristics, and the Institutional setting— can influence the outcomes of 
interest (perceived success, perceived change in staff behavior, fidelity of implemen- 
tation, expected continuation, and dissemination). Each set of factors had different 
effects on implementation and continuation. The rest of this section discusses our 
findings about the effects of these factors. 

Federal Policies. The federal programs affected implementation indirectly 
through the effects of guidelines and management strategies. For example, State 
Title III projects were locally chosen, so that there were often problems arising from 
unclear goals, perhaps because the competitive system of awards tends to encourage 
overly ambitious or general statements of project goals. Federal Title III projects 
were likely to be larger than State Title III projects and more likely to reflect 
superintendents* priorities. This may have led to "top-down" innovation with atten- 
dant difficulties, such as unclear techniques and comparatively little motivation in 
the schools— teachers had no feeling of "ownership" of the projects, as they often did 
for those they had proposed themselves. Title VII bilingual projects were hard to 
implement effectively for reasons that probably reflected the novelty of the federal 
program and its ambitious goals. Projects suffered from inadequate materials, teach- 

4 Federal program effects on incorporation are difficult to summarize or to generalize from our data. 
Right-To-Read specified a management approach that could lead to confusion when unexpected problems 
arose or when teachers could not understand the materials. State Title III was more likely to rely on 
locally developed materials than VEA Part D. State Title III projects were selected by a process that 
allowed the initiation of projects aiming at serious change, often using many of the methods that 
encouraged incorporation; but the same selection process also allowed the initiation of projects that made 
no serious attempt at change. In other words, with respect to incorporation, State Title III was a high-risk, 
high-payoff program. „ * 

The fragmentary nature of our information about federal program effects on incorporation is unfortu* 
nate, because the real test of educational change is whether behavioral change persists at the school and 
classroom level. Continuation decisions do not guarantee incorporation. Therefore, the next phase of our 
research will focus on incorporation for Title III and Title VII. 

5 See Vol W and p. 10 of this volume. 
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er overload, and unrealistic goals or schedules. VEA Part D projects made few 
demands for real change and were relatively easy to implement, although there was 
some parental opposition. But our case studies suggest that host districts treated the 
projects largely as grants-in-aid, and in that situation implementation is unlikely 
to be difficult simply because little or no change is expected. Right-To-Read was 
plagued by implementation problems arising from teacher unfamiliarity with 
materials and lack of prior planning. It also faced occasional parent opposition, 
possibly because parents have a strong interest in reading and may be critical of new 
approaches. 

Federal programs differed little in their effect on project outcomes— fidelity of 
implementation, perceived success, change in staffbehavior, and expected continua- 
tion. 6 There were some significant differences in difficulty of implementatibn: Title 
VII was the hardest to implement and VEA Part D the easiest, but for political and 
social reasons, Title VII projects were nevertheless more likely to be continued than 
VEA Part D projects. 7 

State Title III projects as a whole were no more likely to be continued than other 
kinds of projects, but those that aimed at classroom reorganization and^staff develop- 
ment were more likely to be continued, even though they were often the hardest to 
implement. This may mean that such projects clearly called for behavioral change 
and were unlikely to be initiated from opportunistic motives. 8 

Despite such findings, differences between change agent programs explained 
very little of the differences measured in project outcomes. The other sets of factors 
—institutional setting and project characteristics— accounted for most of the differ- 
ences in outcomes. 

Finally, two broader points about federal policies are in order here. 9 First, many 
of the programs were richer in ambition than in resources— the far-reaching aims 
of Right-To-Read or career education could hardly be met by spending a few million 
dollars a year nationwide. A corollary is that there was probably not much strong 
political support for any of these programs except Title VII. Congress has voted 
constant or declining funds for the other three programs, which in an inflationary 
period is pretty good evidence of political impotence. Federal program officials re- 
sponsible for the smaller programs (Right-To-Read and VEA Part D) have tried to 
turn this adversity into a virtue, pointing out that changes that can be achieved for 
modest sums are more likely to be continued by the schools and adopted by other 
schools after federal seed money is cut off. Our data cannot definitively settle this 
issue. 

. A second general point relates to management. Iii the programs it managed 
directly, the federal government faced more difficult problems in project selection, 
monitoring, and evaluation than SEAs did for programs they managed directly. 

There are thousands of school districts, federal program staffs are small, and 
therefore the abilities of these staffs to evaluate proposals in light of knowledge 
about districts, or, once awards are made, to monitor and evaluate project progress 
are necessarily modest. The difficulties in making good selections are heightened 
when, as for some Right-To-Read projects and first-round VEA Part D projects, 

• See Vol. IV. 

7 See Vol. IV and Sec. V of this volume. 

• For more details about continuation, see Vol. IV, App. C. 

• See the four appendixes to Vol. Ill and Vol. IV, App. B. 
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awards are made on the basis of nominations by states or school districts rather than 
by competitive proposals. The problems of monitoring and evaluating projects once 
an award is made are also heightened by the very nature of these programs. From 
a School district's viewpoint, Right-To-Read school demonstration projects and VEA 
Part D career education projects are one-time grants from small programs. There 
is no particular reason why a district should heed federal project guidelines or 
evaluation results for these programs. 

Furthermore, the very shortage of federal monitoring staffs tended to play down 
the importance of federal objectives in school districts' eyes. Also, direct federal 
management bypassed SEAs, who tended to respond by ignoring the federal projects 
as possible models for dissemination in favor of their own priorities. The federally 
managed programs did maintain some ties to SEAs (Title VII by its political appeal 
and arrangements for SEA review of proposals, Right-To-Read by its SEA program, 
VEA Part D and the federal portion of Title III by the division of managerial 
responsibilities), but apparently they did not carry over into SEA endorsement of 
model projects. 

The chief advantage of SEAs in program management stemmed from their 
geographical and professional "closeness" to LEAs. First, the task was smaller. Even 
a large state, like New York or California, had a limited number of projects to 
manage. For state-managed programs, SEA staff found it easier to visit project sites, 
assist LEAs in project development and work with them in evaluations aimed at 
improving the project as it went along. Second, despite frequent differences of opin- 
ion between SEA and LEA staffs, SEA personnel were more likely to have views that 
were congruent with those of districts that received grants. , 

Despite these advantages, our data show that state-managed programs were 
only slightly more likely to be implemented effectively and continued than federally 
managed ones. 

Therefore, although our findings imply that direct federal management of 
change agent programs at the school district level was less effective than SEA 
management, the policy implications are not conclusive. There may on occasion be 
good reasons for federal agencies to supervise change agent programs directly in 
their early years. For example, if a new program is likely to encounter strong initial 
resistance in some states (Title VII and desegregation aid might be cases in point), 
then the federa' government may wish to build direct links to the constituency 
groups. However, once the constituency is well established, there seems to be a 
strong case for reassigning managerial responsibility to SEAs, under federal super- 
vision. 

Project Characteristics. Project characteristics consisted of four elements: 
(1) educational methods, (2) funding level, (3) implementation strategy, and (4) scope 
of proposed change. The survey and case studies showed that differences in educa- 
tional methods and funding levels (given the fact that funding for these projects was 
small compared to the total budget of the LEA) had little or no effect on differences 
in project outcomes. In other words, an open-classroom project was no more or less 
likely to be effectively implemented than a computer-assisted instruction project, 
nor an expensive project more or less so than a cheaper one. 

The other two project characteristics— -scope of proposed change and implemen- 
tation strategies— did have a marked effect on project outcomes. The major differ- 
ences in substance and scope of change that affected project outcomes were: 
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• Centrality. The closer the goals of a project were to major educational 
objectives of the district, the more likely it was to be continued. 

• Nature and amount of teacher change required. Projects that required 
substantial change in staff behavior (such as classroom reorganization pro- 
jects) often put heavy demands on staff in terms of workload and psycholog- 
ical stress. In the short run teachers were therefore likely to perceive these 
projects as unsuccessful. But, unless the project required such changes in 

, behavior, long-run, effectiveness was suspect. 

• Complexity. (1) If projects tried to cover too many classrooms in a district 
or tried to span elementary and secondary schools, they were not likely to 
be effectively implemented or continued; (2) if the new treatments were 
comprehensive and therefore required broad changes in curriculum or 
significant teacher adaptation, the case studies suggest that they were 
more likely to lead to behavioral change than were narrower treatments; 
(3) if the projects required complex integration of their activities into the 
school program, then they were harder to implement (for example, a 
project with heavy emphasis on teacher training was unlikely to be effec- 
tive unless it also responded to the daily needs of teachers in their class- 
rooms). 

• Consonance. If the values and goals of the project were too dissimilar from 
those of the participants, the project was unlikely to be implemented. 
Effective implementation was normally accompanied by consonance of par- 
ticipants' values with those of the project. 

In terms of policy import, a major finding of this study- involves the character- 
istic of innovative projects that we call implementation stratzgy— the decisions made 
by the local project staff on how to implement their project. We found that the 
innovative strategies selected to carry out a project vitally affected how the project 
turned out. For example, we observed mathematics projects based on the same 
Piagetian ideas and adopted by similar schools in similar districts had different 
outcomes depending on the projects' respective implementation strategies. The rea- 
son why this finding matters for policymakers is .that policy instruments may be 
formulated that could influence the choice of implementation strategies, even 
though there may be no leverage on other aspects of implementation. 

Because a full discussion of various effective and ineffective implementation 
strategies would be too lengthy for this report, we recommend that the reader 
consult other volumes in this study. 10 However, one finding that merits attention 
here involves implementation strategies that promote mutual adaptation and, we 
believe, lead to effective implementation. The following strategies operating to- 
gether promoted mutual adaptation: 

• Adaptive planning. 

• Staff training keyed to the local setting. 

• Local materials development. 

• The establishment of a critical mass of project participants. 

Innovations using these strategies in concert were likely to result in significant 
teacher change that appeared to have been incorporated by the participating staff. 

10 See Vol. IV, Sec. IV; Vol. HI, Sec. HI; and Vol. II, Sec. IV. 
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The Institutional Setting. The local institutional setting was by far the most 
important factor in determining project outcomes, according to both the survey data 
and the case studies. The most important elements were high teacher morale and 
willingness to do extra work, and the support of principals, as well as of district 
officials. The reasons seem evident. If teachers see that they are getting support from 
each other and from administrators, they are being told that they can "afford" to 
take the project seriously. This also makes mutual adaptation easier because it 
encourages the kind of changes that allow the institution to adapt to the project. 
Successful implementation was unlikely where the setting was unreceptive or indiff- 
erent (as was frequently the case in high schools, where the professional departmen- 
tal structure — English, math, social studies, etc.— inhibits broad organizational 
change). These findings relate to our results about successful implementation strate- 
gies. Adaptive planning, staff training keyed to the local setting, and local materials 
development all helped to secure the support and commitment of school and district 
staffs. 



CONCLUSIONS 

1. Federal program policies were directly important only at the initiation stage;, 
even at that stage, differences among change agent programs' policies made 
little difference to project design choices. 

2. Direct federal selection and monitoring of change agent projects at the school 
district level were generally somewhat less effective than SEA management. 

3. The extent to which the LEA initiation behavior was characterized by prob- 
lem-solving motives or opportunistic ones had pervasive influence on effective 
implementation and continuation. 

4. A receptive local institutional setting was necessary, but not sufficient for 
effective implementation. 

5. Two kinds of project characteristics— an effective implementation strategy 
and a scope of change that was as broad as- the setting would reasonably 
allow— were important in promoting mutual adaptation and successful out- 
comes. Other project characteristics (differences in technology or resources) 
made little difference to project outcomes, which were primarily shaped by 
local factors. 

6. Mutual adaptation and successful implementation outcomes were no guaran- 
tee of project continuation after federal funding stops. The decision to contin- 
ue was affected by project success, but costs and political or bureaucratic 
acceptability were probably of equal importance. 
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V. POLICY IMPLICATIONS? 



This section, like the corresponding section of Vol. IV, is tentative about the 
implications of the work for federal policy. Once the second phase of the Rand study 
is done (in late 1976), we will have more evidence about what happens to projects 
after outside funding is gone. 

Furthermore, it is important to remember that th6 data and findings of this 
study are largely based on what happens to projects, and not on the social effects 
of the programs. Congress and the Executive Branch supported each program for 
different reasons— for example, general encouragement of innovations in the case 
of Title III, or establishing a high priority for reading in the case of Right-To-Read. 
The Rand first-year study focuses on projects — on what factors affect their outcomes 
and on the process of change that occurs, or fails to. This approach can lead to or 
stem from the belief that a program's effects are largely measurable by the effects 
of its constituent projects. But there is more to it than that. Title III or Right-To- 
Read or any other program may have effects that are important no matter what 
happens to individual projects. For example, the Rand study indicates that among 
the four programs, bilingual education projects (Title VII) are the hardest to imple- 
ment and are the least successful in meeting their goals. Nevertheless, as of 1975, 
Title VII is the only one of the programs that Congress is willing to support with 
more and more money each year. Title VII projects on the average may not be very 
effective by the standards of efficiency in innovation, but the program has been most 
effective in legitimizing Spanish-speaking people's demands that the schools pay 
more attention to their children's needs. In a sense, efficiency has nothing to do with 
it. There might be cheaper and more effective ways to meet the needs at which 
bilingual programs aim. But, the political test is potency— the ability of the claim- 
ants to win large-scale support from Congress, and thereby the political respect, 
however reluctant, of school districts that formerly could igndre their demands. 
Forcing the districts to teach in Spanish is a test of that potency, and thereby 
contributes to the transcending aim—increasing the social self-respect and political 
power of Mexican-Americans and Puerto Ricans, children and adults alike. It is 
important, therefore, to remember the limitations of this study as an assessment of 
federal change agent programs. These larger issues are beyond our scope, and 
mostly beyond the scope of any scientific research on effectiveness. 

This view is consistent with our finding that federal programs' effects on educa- 
tional innovations are chiefly important for the priorities they establish among 
categories. For example, existing change agent programs promote LEA activity in 
reading, bilingualism, or innovative practices. But if change agent programs focused 
instead on the arts, the three Rs, or gymnastics, then the distribution of change 
agent funds among districts would presumably change, as would the priorities of 
many districts. 

To the extent that federal policy follows a rational research and development 
model of educational change, it may tq some extent overlook the major findings of 
the first year's work, which tend to contradict that model. These findings were: 

• Project outcomes were primarily determined by the local institutional 
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setting; in that setting, implementation dominated outcomes, mutual adap- 
tation was the basis of lasting change, and a receptive institutional setting 
* was a necessary condition for mutual adaptation. 

• Technology was not self-winding; once instituted, it could not be automati- 
cally implemented according to preset plans. 

• Districts had different capacities to innovate and to deal with different 
stages of the innovative process. 

• Federal change agent policies had little effect on project outcomes. 

If these findings are correct, then they imply that USOE might well seek to 
explore changes in federal policies toward LEA innovation. Such explorations might 
include: (1) funding projects according to successive stages of development in the 
organizational setting; (2) formulating program guidelines to promote mutual adap- 
tation; (3) using federal incentives to stimulate institutional capacity for problem- 
solving. Volume IV discusses how such initiatives might be explored, pointing out 
Ihe difficulties and uncertainties involved in following these lines, at least on the 
basis of vrnat we know now. For example, if the circumstances influencing the 
decision to initiate a project turn out to affect decisions about continuation, what are 
the implications for federal policies? Or, considering how strongly the institutional 
setting affects the district's capacity to change, how can federal policies be made 
flexible enough to take account of real differences among districts? 

Finally, the reader should be warned that there are ; mplicit assumptions that 
underlie the request for this study and therefore shape the direction of the research 
— that educational reforms are desirable, and that a more effective change process 
is called for. There is also an assumption in these tentative policy implications that 
many people may question: namely, that federal policies should "bring districts 
along" so that they can innovate more effectively. Behind this assumption rests a 
tradition which implies that federal aid should be used to accomplish efficiently the 
aims of national policy. But this conflicts with another tradition— one that rejects 
conscious federal direction of American public schools and finds positive virtue in 
policies that are "ineffective" in imposing federal goals on local schools. Our findings 
about the importance of centrality and consonance 1 attest to the strength of thj* 
tradition — federal agencies propose, LEAs dispose. If our recommendations should 
lead to federal policies that successfully promote systematic changes in LEA atti- 
tudes and capacities for innovation, the results may not satisfy educators, parents, 
students, and the community at large. These observations are not a subject of this 
research, but they are part of the context for considering any policy recommen- 
dations that may flow from this study. 

1 Vol. IV and p. 15 of this volume. % 
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